The U.S. Security Threats Dataset

This dataset uses a text mining approach to analyze the Annual Threat Assessment (aka, Worldwide Threat Assessment) of the U.S. Intelligence Community. 

The reports can be found on the Office of the Director of National Intelligence (ODNI)'s website: https://www.dni.gov/index.php/newsroom/reports-publications/reports-publications-2022/item/2279-2022-annual-threat-assessment-of-the-u-s-intelligence-community. 

Importantly, in downloading the replication files from the Harvard Dataverse, the Threat Assessments, which are txt files, should download in a separate folder named "ta". You should leave them in this folder to run the replication code properly. If they download separately for some reason, please create a new folder named "ta" and put the Threat Assessment txt files in that folder.  

Additionally, please note that the report was not released in 2020. Therefore, I use the 2019 report for 2020 as a simple imputation. 

Method: To generate the security threat perceptions dataset, I tabulate all state references, which include state names, nationalities, and capital cities, across the Assessments published to date. I tabulate capital cities and nationalities, in addition to state names, because the Assessments refer to states in a few different manners. For example, “Russia,” “Russian,” and “Moscow” all essentially have the same meaning within the context of the Assessments. After tabulating all state references, I generate a state’s Threat Share. The Threat Share is the number of times a state is referenced in a given year as a percent of all state references in a given year.

It is important to clarify that the Threat Share metric captures the sources of security threats. A source may be a government that the IC finds threatening, but it need not be. To illustrate this nuance, consider Iraq and Afghanistan over the last decade. These states are high sources of security threats, even though their de jure governments have not necessarily been threatening to the United States in recent years.

Jeff Allen (jeffreysallen1@gmail.com). 